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ABSTRACT 


This  paper  explores  the  potential  use  of  Reliability  Growth 
Curves  as  a means  by  which  the  program  manager  can  monitor  and 
measure  the  reliability  growth  of  a weapon  system.  The  present 
reliability  policies  associated  with  weapon  system  acquisition 
are  reviewed.  Examination  of  three  weapon  system's  Request  for 
Proposals  show  how  these  policies  are  implemented.  MIL-STD 
785A,  which  provides  the  guidelines  for  reliability  programs , 
is  also  reviewed  and  is  followed  by  a discussion  of  the  weak- 
nesses of  today's  policies.  This  sets  the  stage  for  the  pre- 
sentation of  the  proposed  method  that  will  enable  the  program 
manager  to  control  the  reliability  growth  of  new  weapon  systems. 
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I. 


INTRODUCTION 


A.  BACKGROUND 

Programs  for  the  acquisition  of  weapon  systems  are  con- 
stantly facing  new  challenges.  This  is  the  result  of  infla- 
tion, rising  costs  and  pressures  both  from  Congress  and  the 
public.  Elmer  B.  Staats,  Comptroller  General  of  the  United 
States  in  1974,  states  [1]: 

"The  rising  concern  of  the  Congress  and  the  public  about 
increasing  costs  of  new  defense  weapons  is  certainly  not 
news.  The  patience  of  the  public  and  the  Congress  has 
been  worn  thin  by  repeated  experiences  with  contractor 
and  Defense  Department  cost  forecasts  for  new  weapons 
that  are  unrealistic,  and  the  promises  of  performance 
miracles  by  new  weapons  that  later  proved  unattainable." 

In  addition,  today’s  defense  strategies  and  tactics, 

designed  to  meet  the  rapidly  changing  international  situations, 

require  weapon  systems  of  increasing  complexity  which  demand 

an  increase  in  the  performance  requirements  and  environmental 

operating  conditions.  This  requirement  for  complex  weapon 

systems  has  created  a technology  explosion  which  threatens  to 

doom  systems  under  development  to  technical  obsolescence 

before  they  become  operational.  These  problems  were  cited  in 

1972  by  former  Deputy  Secretary  of  Defense,  David  Packard  [2]: 

"Cost  overruns  were  the  most  visible  symptoms  of  the 
troubled  new  weapon  development  situation  but  there  were 
other  problems  too.  Most  programs  took  far  too  long  from 
original  conception  until  weapons  were  delivered  to  the 
operational  military  forces.  As  a result,  many  weapons, 
particularly  those  involving  electronics  and  other  fast- 
moving  technology,  were  out  of  date  by  the  time  they  were 
available. " 

To  compound  the  problem,  even  after  exorbitantly  high  cost 
and  unnecessarily  long  development  time,  many  of  the  new 
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devices  did  not  have  the  reliability  that  is  needed  for  mili- 
tary use. 

With  these  various  factors  affecting  the  acquisition  of 
• weapon  systems,  how  does  one  achieve  a highly  reliable  weapon 

system?  Management  in  the  Department  of  Defense  and  contrac- 
tors have  increased  their  efforts  to  improve  the  reliability 
of  weapon  systems.  However,  despite  these  efforts,  the  reli- 
ability of  weapon  systems  continues  to  be  disappointing. 
Studies  throughout  the  Department  of  Defense  have  been  and 
continue  to  be  made  in  an  effort  to  find  ways  to  improve 
reliability . 

In  May  1972,  the  "Mean  Time  Between  Failure"  Improvement 
Study  Group,  headed  by  RADM  K.  R.  Wheeler,  SC  USN,  conducted 
a major  study  for  the  Navy  on  reliability  and  maintainability 
improvement.  The  resulting  report,  better  known  as  the 
"Wheeler  Report"  [3]  concluded  that  those  equipments  found 
unreliable  in  the  fleet  simply  did  not  have  reliability 
designed  in  from  the  beginning.  The  report  cited  the  follow- 
ing specific  conditions  existing  in  1972  which  were  considered 
to  be  crucial  factors  influencing  reliability  in  Navy  acquisi- 
tion efforts. 

Sufficient  time  and  dollars  omitted  from  budgetary 
submissions 

Unenforcible  reliability  goals  in  contracts  and  no 
enforcible  reliability  requirements 

Poor  or  nonexistent  reliability  testing  procedures 

Insufficient  reporting  systems  to  identify  unreliable 
equipments  in  the  fleet 

Pressure  for  attaining  specified  performance  goals 
which  lead  to  trade-offs  that  though  not  explicit, 
aggregate  to  lower  reliability 
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Poor  documentation  of  trade-offs  between  reliability 
and  performance 

In  1973  the  Air  Force  also  conducted  a study  [4]  in  which 
they  found  37  possible  deficiencies  that  could  affect  reli- 
ability of  the  weapon  systems.  The  following  is  a listing  of 
those  that  have  a relation  to  the  acquisition  of  weapon 
systems : 

Contractor  has  no  incentive  to  overdesign  reliability 
in  equipment 

No  provisions  made  for  reliability  growth  during 
development 

Changes  in  mission  profiles  of  operational  equipment 

Operational  equipment  used  in  environments  for  which 
they  were  not  designed 

Contract  reliability  requirements  changed  to  goals 
or  not  enforced 

As  a result  of  these  Governmental  studies,  new  reliability 
policies  were  issued.  One  emphasis  being  that  quantitative 
reliability  requirements  will  be  stated  in  "Request  for  Pro- 
posals" (RFP) . The  policies  further  state  that  these  require- 
ments will  be  minimum  acceptable,  realistic  and  achievable. 

The  effects  of  these  new  policies  are  still  questionable.  In 
the  article  "Improving  R & D Management  Through  Prototyping," 
David  Packard  states  [5]: 

"A  serious  problem  that  troubles  all  of  our  recent  major 
programs  is  reliability.  Numerous  directives,  specifica- 
tions, and  other  requirements  have  been  placed  on  all 
major  development  programs  to  attempt  to  improve  the  reli- 
ability of  new  weapons.  Very  little  improvement,  if  any, 
has  come  from  this  effort  and  very  large  sums  of  money 
have  been  spent. 

"Reliability  cannot  be  achieved  by  adhering  to  detailed 
specifications.  Reliability  cannot  be  achieved  by  formula 
or  by  analysis.  Some  of  these  may  help  to  some  extent  but 
there  is  only  one  road  to  reliability.  Build  it,  test  it, 
and  fix  the  things  that  go  wrong.  Repeat  the  process  until 
the  desired  reliability  is  achieved.  It  is  a feedback  pro- 
cess and  no  other  way." 
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B.  PURPOSE 


Reliability  has  become  an  issue  of  major  concern  in  the 
acquisition  of  new  weapon  systems  and  today's  reliability 
policies  do  not  provide  the  program  manager  with  the  needed 
visibility  during  the  weapon  system's  development.  The  pur- 
pose of  this  paper  is  to  propose  a method  by  which  the  program 
manager  can  monitor  and  measure  the  reliability  growth  of  the 
weapon  system  as  it  proceeds  through  its  life  cycle  from 
initial  design  to  the  end  of  development. 
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II.  WHY  A RELIABILITY  PROGRAM 


One  of  the  major  forces  that  demands  a reliability  program 
is  the  ever  increasing  cost  of  weapon  systems.  This  is  well 
recognized  by  the  Department  of  Defense  as  evident  by  their 
emphasis  on  the  "Design  to  Cost"  concept  of  which  reliability 
is  an  essential  element.  This  is  further  amplified  by 
CDR  Hollister,  USN,  and  R.  Shorey  of  the  Office  of  the  Assist- 
ant Director  (Electronics)  [6]: 

"In  design  to  a cost  development,  reliability  and  maintain- 
ability are  especially  important  considerations.  There  is 
concern  that  a developer,  motivated  to  provide  the  highest 
level  of  system  performance  possible  at  a target  cost,  may 
implicitly  trade  reliability  and  maintainability  for  greater 
performance. . . 

"Minimum  reliability  and  maintainability  requirements  are 
established  along  with  other  minimum  acceptable  performance 
specifications.  These  are  treated  as  major  system  perform- 
ance parameters  during  performance  verification." 

In  his  book  "Reliability  Mathematics,"  Bertram  L.  Amstradter 
states  [ 7 ] : 

"The  importance  of  obtaining  highly  reliable  systems  and 
components  has  been  recognzied  in  recent  years.  From  a 
purely  economic  viewpoint,  high  reliability  is  desirable 
to  reduce  overall  costs.  The  disturbing  fact  that  the 
yearly  cost  o'f  maintaining  some  military  systems  in  an 
operable  state  has  been  as  high  as  ten  times  the  original 
cost  of  the  equipment  emphasizes  this  need... 

"The  need  for  and  importance  of  reliability  have  been 
reflected  in  the  constantly  increasing  emphasis  placed  on 
reliability  by  both  government  and  commercial  industry." 

Other  factors  that  demand  reliability  improvement  are 
safety,  schedule  delays,  inconvenience,  and  combat  effective- 
ness in  the  fleet.  Willis  J.  Willoughby,  Deputy  Chief  of 
Naval  Material  for  Reliability  and  Maintainability,  states 
[8]: 
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"The  Navy,  like  other  Military  Services,  is  faced  with 
less- than-desired  combat  effectiveness  levels  and  rising 
life  cycle  costs.  This  situation  exists  because  reli- 
ability has  not  been  properly  specified  and  pursued; 
rather,  the  Navy  has  relied  heavily  on  integrated  logistic 
support. " 
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III.  PRESENT  RELIABILITY  POLICIES 


A.  POLICIES 

Various  instructions  that  have  been  issued  provide  the 
best  overview  of  the  present  policies  governing  reliability 
and  the  acquisition  of  weapon  systems. 

1.  SECNAVINST  5000.1 

SECNAVINST  5000.1,  System  Acquisition  in  the  Depart- 
ment of  the  Navy,  was  issued  in  March  1972  and  contains  as 
Enclosure  (1),  DOD  Directive  5000.1.  This  instruction  now 
establishes  all  major  policy  and  management  principles  for 
the  acquisition  of  weapon  systems.  In  spite  of  the  fact  that 
reliability  is  now  a major  issue  in  the  acquisition  of  weapon 
systems,  no  reference  to  reliability  can  be  found  in  the  40 
pages  of  this  instruction.  In  Section  II,  Conduct  of  Program, 
reference  is  made  to  Enclosure  (4)  of  the  instruction  which 
is  a list  of  applicable  documents.  This  list  contains  no  more 
than  56  different  instructions  that  have  some  bearing  on  the 
acquisition  of  weapon  systems.  Buried  in  this  list  is 
SECNAVINST  3900. 36A,  which  is  the  instruction  governing  Reli- 
ability and  Maintainability. 

2.  SECNAVINST  3900, 3 3A 

SECNAVINST  3900. 33A,  Initiation  of  Engineering  and 
Operation  Systems  Development,  was  issued  in  September  1968. 
This  instruction  established  the  Navy  policies  for  the  acqui- 
sition of  weapon  systems  during  the  Conceptual  Formulation 
and  Contract  Definition  phases.  The  following  statements 
pertaining  to  reliability  were  contained  in  this  instruction: 
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System  Trade-offs:  Trade-offs  will  be  used  to  obtain 


optimum  balance  between  total  cost,  schedule,  and  oper- 
ational effectiveness  (reliability)  for  the  system. 

Conceptual  Phase:  Quantitative  reliability  and  maintain- 

ability goals  and  demonstration  concepts. 

Engineering  Development:  Quantitative  reliability  and 

maintainability  specifications  for  the  system  and  major 
subsystems  and  proposed  test  plans  to  demonstrate  their 
achievement. 

This  instruction  has  been  superseded  with  the  issuance  of 
SECNAVINST  5000.1. 

3.  QPNAVINST  5000. 4 2 A 

OPNAVINST  5000. 42A,  Weapon  Systems  Selection  and 
Planning,  was  issued  in  March  1976.  This  instruction  estab- 
lishes the  policies  for  identifying  operational  requirements 
(OR)  and  conducting  management  reviews  during  system  acquisi- 
tion. The  instruction  makes  only  two  references  to  reliability. 
One  being  that  reliability  is  to  be  considered  in  establishing 
system  performance  goals.  The  second  reference  is  the  follow- 
ing statement: 

From  the  outset,  planning  will  accord  high  priority  to 
simplicity  in  design  and  toughness  of  management,  including 
trade-offs  and  contractual  provisions,  to  ensure  a high 
degree  of  reliability  and  maintainability. 

4.  SECNAVINST  3900. 36A 

SECNAVINST  3900. 36A,  Reliability  and  Maintainability 
(R  & M)  for  Naval  Material,  was  issued  in  June  1970.  This 
instruction  establishes  the  R & M policies  for  the  Navy  and 
assigns  responsibility  for  its  achievement.  With  regards  to 
the  acquisition  of  weapon  systems,  the  following  is  found  in 
this  instruction: 

Basic  Concept:  Reliability  techniques  are  based  on  the  fact 

that  failure  will  occur  and  they  focus  attention  on  means 
to  minimize  the  effect  of  failure.  The  required  R & M is 
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justified  in  that  it  supports  mission  requirements. 

Therefore,  it  must  be  considered  throughout  the  life 
cycle  of  the  system. 

Policy:  During  the  conceptual  and  development  phases  of 

new  systems,  each  program  will  include  a minimum  acceptable 
R & M value  based  on  operational  need.  A specified  confi- 
dence level  is  normally  required  for  the  R & M demonstration. 

Responsibilities : 

a.  The  Chief  of  Naval  Operations  is  responsible  for; 

Ensure  that  requirements  documents  include  numerical  R & M 
requirements  and  "Goals"  or  "Objectives"  shall  not  be  used 
in  lieu  of  R & M requirements. 

Evaluate  proposed  decreases  in  R & M requirements  for  impact 
on  operational  characteristics  where  development  is  unable 
to  economically  achieve  R & M minimum  values. 

b.  The  Chief  of  Naval  Material  is  responsible  for; 

Ensure  that  quantitative  R & M requirements  are  responded 
to  in  proposal  for  industry. 

Determine  adequacy  of  each  contractor's  R & M program  for 
achieving  and  demonstrating  R & M requirements. 


B.  REQUEST  FOR  PROPOSALS 

To  obtain  a first  hand  knowledge  of  how  these  policies 
are  implemented,  three  Request  for  Proposals  (RFP)  were 
reviewed. 

1.  Harpoon  Weapon  System  [9] 

a.  Reliability  Standards  listed 

(1)  MIL-STD  721:  Definition  of  Effectiveness  Terms 

(2)  MIL-STD  756:  Reliability  Prediction 

(3)  MIL-STD  781:  Reliability  Tests 

(4)  MIL-STD  785:  Reliability  Program  Plan 

(5)  MIL-STD  1304A:  Reliability  Reports 

(6)  MIL-R-22732 : Reliability  Requirement  for 

Shipboard  and  Ground  Electrical 
Equipment 

b.  Reliability  Section 

(1)  Definition:  A definition  for  reliability  was 

provided  in  the  RFP  and  it  was  the  one  cited  in  MIL-STD  721. 


15 


r 


(2)  Reliability  Requirements:  The  requirements 

stated  in  the  RFP  were  of  two  types,  a reliability  probability 
and  a Mean  Time  Between  Failure  (MTBF) . Numerical  numbers 
were  stated,  however,  in  the  listing  below  only  the  reliability 
probability  (R)  or  the  MTBF  will  be  noted. 


Sub  System 

Min  Acceptable 
Reliability  Req't 

Reliability 
Design  Objective 

Missile 

Freeflight 

R1 

R2 

Standby 

MTBF^ 

MTBF  2 

Ready 

MTBF1 

MTBF  2 

Storage 

R1 

R2 

Aircraft  Command 
and  Launch 

MTBF1 

MTBF  2 

Shipboard  Command 
and  Launch 

MTBF 

MTBF  2 

The  RFP  further  amplifies  the  requirements  by  providing  the 
following : 

Missile  Freef light  Reliability:  They  shall  have 

a freeflight  reliability  of  R2  as  a design  objective  and  Ri 
as  a minimum  acceptable  value.  The  missile  shall  be  able  to 
operate  under  a degraded  mode.  Free-flight  is  defined  as  the 
missile  profile  from  launch  to  target  intercept. 

Aircraft  Equipment  Reliability:  Each  new  item  of 

equipment  for  aircraft  command  and  launch  shall  have  a MTBF 
(numerically  specified)  design  goal  and  a MTBF  (numerically 
specified)  minimum  acceptable  value. 

Shipboard  Equipment  Reliability:  Same  type  of 

statement  as  for  the  aircraft  equipment. 

(3)  Reliability  Program  Plan:  With  regards  to 

a reliability  program  plan,  the  RFP  states  the  following: 

The  contractor  shall  provide  and  maintain  a reli- 
ability program  that  is  acceptable  to  the  government. 

The  contractor  shall  use  design  techniques 
described  in  paragraph  5.2.1  of  MIL-STD  785A  to  the  extent 
practicable  during  the  design  phase  to  assure  that  the  reli- 
ability requirements  will  be  met  during  the  weapon  systems 
development  phase. 
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The  contractor  shall  conduct  a reliability  analysis 
of  the  Harpoon  Weapon  System  during  the  design  phase  in  accord- 
ance with  paragraph  5.2.2  and  all  subparagraphs  of  MIL- STD  785A. 

2.  VFAX  Airborne  Weapon  System  [10] 

a.  Reliability  Standards  listed 

(1)  MIL-STD  721:  Definition  of  Effectiveness  Terms 

(2)  MIL-STD  756:  Reliability  Prediction 

(3)  MIL-STD  785:  Reliability  Program  Plan 

b.  Reliability  Section 

(1)  Reliability  Requirements:  Quantitative 

requirements  are  stated  for  two  phases  of  the  weapon  system. 

Mission  reliability:  The  mission  reliability, 

expressed  as  the  probability  that  the  Airplane  Weapon  System 
can  perform  all  the  mission  functions  successfully,  shall  equal 
or  exceed  O.XX  as  a goal. 

Refly  reliability:  The  refly  reliability,  expressed 

as  the  probability  that  the  Airplane  Weapon  System  can  be  re- 
turned to  full  operating  capability  without  corrective  main- 
tenance between  missions,  shall  equal  or  exceed  O.XX  based  on 
a t-hour  mission  duration  with  O.XX  as  a goal. 

(2)  Reliability  Program  Plan:  The  request  for 

proposal  states  that  a reliability  program  will  be  established 
in  accordance  with  MIL-STD  785  and  the  reliability  specifica- 
tion appendix  of  the  request  for  proposal.  The  appendix  pro- 
vides amplification  of  various  sections  of  MIL-STD  785  and 

in  some  cases  specific  instructions  as  to  how  and  what  will 
be  included  in  the  program. 

A section  of  the  reliability  specification 
appendix  states  what  testing  is  required  demonstrating  the 
reliability  of  the  weapon  system.  The  tests  are  classified 
in  two  categories,  preproduction  and  quality  conformance. 

The  preproduction  test  is  to  determine  if  the 
Airplane  Weapon  System  has  met  the  quantative  reliability 
requirements.  Whereas,  the  quality  conformance  test  is  to 
determine  if  the  Airplane  Weapon  System  offered  for  acceptance 
has  met  the  quantitative  reliability  requirements. 

3.  AEGIS  Weapon  System  [11] 

a.  Reliability  Standards  listed 

(1)  MIL-STD  721:  Definition  of  Effectiveness  Terms 

(2)  MIL-STD  756:  Reliability  Prediction 

(3)  MIL-STD  781:  Reliability  Tests 
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(4)  MIL-STD  785:  Reliability  Program  Plan 

(5)  MIL-R  22732:  Reliability  Requirements  for 

Shipboard  and  Ground  Electrical 
Equipment 

b.  Reliability  Section 

(1)  Definitions:  The  RFP  provides  definitions 

for  System  Effectiveness,  Inherent  Availability  and  Operational 
Availability. 

(2)  Reliability  Requirements:  The  requirements 

stated  in  the  RFP  are  of  two  types,  those  being  availability 
and  reliability. 

Shipboard  System  Availability:  The  minimum  accept- 

able shipboard  system  inherent  availability  shall  be  at  least 
O.XXX  over  a t-month  deployment  period  under  the  following 
conditions : 


(a)  The  system  will  be  in  the  standby  or 
operate  mode  of  operation  at  least  75  percent  of  the  time. 

(b)  The  MFAR  radar  is  to  be  considered  a 
primary  search  radar. 

(c)  The  system  is  considered  available  when; 
It  is  down  for  preventive  maintenance,  provided  the  capability 
to  return  to  full  operation  within  t-minutes  is  retained. 

Five  out  of  six  illuminators  are  available  and  remainder  of 
the  system  segments  are  providing  specified  performance. 

Shipboard  System  Reliability:  The  MTBF  for  the 

system,  including  test  sets  and  built-in  monitoring  devices, 
but  excluding  missiles  shall  be  not  less  than  t-hours . 

(3)  Reliability  Program  Plan:  The  reliability 

program  plan  requirement  has  been  divided  into  three  phases . 

CD  Phase  A:  The  contractor  shall  submit  as  part 

of  his  proposal  a Preliminary  Reliability  and  Maintainability 
Plan  identifying  the  work  to  be  performed  during  Engineering 
Development  (ED)  and  its  relationships  to  and  expected  impact 
on  the  designs  formulation  process. 

CD  Phase  B:  The  contractor  shall, during  this 

phase,  expand  the  plan  into  a comprehensive  program  for  execu- 
tion during  engineering  development.  This  program  plan  shall 
comply  with  requirements  of  MIL-STD  785  and  MIL-STD  470.  The 
plan  shall  present  detailed  procedures  by  which  reliability 
and  maintainability  will  be  measured,  evaluated,  and  con- 
trolled throughout  ED. 

ED  Phase:  The  contractor  shall  develop  a Produc- 

tion R & M Plan.  This  plan  shall  establish  and  describe 
those  detailed  procedures  by  which  inherent  R & M character- 
istics are  monitored  and  retained  during  production. 
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The  RFP  further  breaks  down  the  reliability  pro- 
gram to  detailed  task  requirements  as  follows: 

Requirements  Interpretation,  Allocation  and 
Feasibility  Analysis 
Predictive  Assessments 
Design  Selection 
Design  Analysis 

Parts  and  Materials  Selection  and  Control 

Parts  Qualification 

Reliability  Quality  Assurance 

R & M Data  Reporting,  Analysis  and  Feedback 

Design  Qualification  Tests 

Acceptance  Tests 

(4)  Policies:  To  obtain  maximum  availability 

without  sacrificing  any  of  the  high  performance  requirements, 
the  following  policies  have  been  set  forth: 

Maximum  utilization  of  proven  components--the 
design  trade-offs  must  favor  proven  reliability  over  potential 
savings  in  weight,  space  or  costs  of  unproven  components. 

Standardization — Standardization  of  components, 
printed  circuit  cards,  modules  and  subassemblies  must  be 
maximized. 


Redundancy  and  Fault  isolators — the  system  design 
must  be  optimized  for  maximum  reliability  and  minimum  down 
time  through  the  use  of  redundancy,  automatic  systems  monitor- 
ing and  simplified  fault  isolators. 


C.  MIL-STD  785A 

Title:  Reliability  Program  for  Systems  and  Equipment 

Development  and  Production. 

Purpose:  To  establish  uniform  criteria  for  a reliability 

program  and  provide  guidelines  for  the  preparation  and  imple- 
mentation of  a reliability  program. 

Outline:  This  standard  provides  guidelines  as  to  what 

shall  be  included  in  a reliability  program  plan.  Major  head- 
ings that  are  to  be  in  this  program  plan  are: 

Reliability  Management 
Reliability  Design  and  Evaluation 
Reliability  Testing  and  Demonstration 
Failure  Data 
Production  Reliability 
Status  Reports 


19 


As  one  reviews  these  various  sections  of  the  MIL-STD,  it 


is  noted  that  the  context  is  in  generalities  and  no  specific 
detailed  guidance  is  given.  In  the  Program  Review  section 
; (5.1.4)  and  the  Design  Review  sections  (5.2.7),  it  states  that 

progress  will  be  reviewed  at  appropriate  stages  of  development 
and  production  to  evaluate  achievement  of  the  reliability 
requirements.  However,  no  information  is  given  as  to  how  the 
progress  will  be  monitored  or  to  what  criteria  it  is  to  be 
measured  against,  other  than  the  numerical  requirement  (which 
is  the  number  to  be  achieved  at  the  end  of  development) . 

D.  SUMMARY 

J 

A review  of  the  policies  has  shown  that  very  little  guid- 
ance is  given  in  terms  of  reliability  and  the  acquisition  of 
a weapon  system.  Guidance  can  be  summed  up  in  the  following 
statement: 

A minimum  acceptable  numerical  requirement  and  a Reliability 
Program  requirement  will  be  stated  in  the  RFP. 

The  three  RFP's  that  were  reviewed  showed  that  this  is 
exactly  what  was  stated.  As  to  the  completeness  and  detail 
of  the  requirements,  this  depends  upon  the  program  manager 
who  is  responsible  for  the  context  of  the  RFP,  as  can  be  seen 
by  the  variance  in  the  RFP's.  In  each  of  the  RFP's,  three  to 
six  military  standards  are  referenced,  but  only  one  (MIL-STD 
785A)  is  cited  in  the  reliability  section.  In  the  VFAX  and 
AEGIS  RFP's,  additional  detail  is  given  to  supplement  MIL- 
STD  785A.  However,  there  is  no  information  given  in  any  of 
the  three  RFP's  as  to  how  the  reliability  of  the  system  will 
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be  monitored  or  measured  during  its  development.  As  noted  in 
the  review  of  MIL-STD  785A,  which  is  cited  in  all  the  RFP's, 
no  information  is  contained  as  to  monitoring  or  measurement 
of  reliability  during  development. 


MANAGEMENT  AND  RELIABILITY  GROWTH 


i 
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A.  INTRODUCTION 

'Reliability  Prediction"  can  be  defined  as  an  estimate  of 
the  performance  we  can  expect  of  the  equipment  on  its  origi- 
nally scheduled  delivery  date. 

It  is  essential  in  treating  the  subject  of  reliability  to 
understand  basic  concepts  relating  to  techniques  for  predic- 
tion and  control  of  reliability.  The  following  contributions 
from  persons  knowledgeable  in  the  field  are  intended  to  pro- 
vide this  understanding. 

"The  formal  process  of  Reliability  Prediction  is  based  on 
the  exponential  distribution  of  random,  statiscally  inde- 
pendent failures,  occurring  as  a result  of  some  inherent 
nonsystematic  propensity  of  things  to  fail.  The  mathematics 
associated  with  this  process  requires  that  failure  rates  of 
each  constituent  item  be  known  and  constant  with  time.  The 
resulting  failure  rate  "prediction"  is  independent  of  time. 
This  is  a fatal  flaw,  because  we  observe  achieved  failure 
rate  to  be  a function  of  time.  That  being  the  case,  the 
prediction  can  only  represent  an  ultimate  limiting  value. 
Such  a value  might  be  of  some  use  if  it  could  be  depended 
upon,  but  it  cannot,  because  performance  is  exceeding  pre- 
dicted values  every  day... 

"The  numbers  game  has  outlived  its  usefulness  to  the  point 
of  severely  interfering  with  our  ability  to  do  the  real 
job.  In  today's  explosive  development  of  new  parts,  and 
equipment  of  astonishing  capability — where  successive  models 
are  each  a new  generation  of  development — classical  reli- 
ability prediction  is  nearly  useless:  The  numbers  it  pro- 

duces are  rarely  relevant  to  the  solution  of  any  of  our 
problems."  (Ernest  Codier,  General  Electric  Co.)  [12] 

"Many  popular  techniques  for  the  analysis  of  reliability 
consider  the  problem  only  at  a single  point  in  time.  Such 
techniques  certainly  yield  value  information.  However,  a 
complete  treatment  of  system  reliability  requires  careful 
consideration  of  the  time  variations  introduced  by  design 
changes  or  modifications  in  maintenance  practices." 

(J.  T.  Duane,  General  Electric  Co.)  [13] 
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"Experience  has  shown  that  reliability  values  reported  from 
field  service  are  generally  lower--often  much  lower — than 
those  predicted  by  design  analysis  or  demonstrated  by  labor- 
atory testing.  This  has  led  to  optimistic  projections  of 
operational  effectiveness  and  ownership  cost.  It  can  also 
lead  managers  down  a "primrose  path"  during  development, 
since  predictions  and  demonstrations  indicate  that  reli- 
ability requirements  are  being  met."  (Robert  Parker,  Deputy 
Director  Defense  Research  and  Engineering)  [14] 

"...Despite  special  management  efforts  and  major  techno- 
logical improvements.  Air  Force  avionics  equipment  still 
does  not  demonstrate  high  reliability.  This  shortcoming 
is  compounded  by  the  fact  that  the  reliability  predictions 
which  are  formulated  during  equipment  design,  refined  during 
development,  and  tested  during  formal  laboratory  demonstra- 
tions, infer  equipment  reliability  that  is  often  signifi- 
cantly greater  than  what  is  actually  achieved  when  the 
equipment  is  operationally  employed."  (Lt  Gen  Robert  Marsh, 
USAF,  Vice  Commander,  Air  Force  Systems  Command)  [15] 

As  noted  by  the  various  comments,  the  conventional  reli- 
ability analysis  techniques,  in  particular  "reliability  pre- 
diction," are  not  providing  the  desired  reliability  in  weapon 
systems.  Reliability  prediction  provides  a useful  data  point. 
However,  it  is  being  used  in  dealing  with  time  and  design 
changes,  but  it  does  not  provide  the  manager  with  a useful 
tool  to  monitor  the  reliability  growth  of  the  weapon  system 
during  its  development.  In  the  implementation  of  the  present 
* reliability  policies  (Section  III) , the  manager  is  in  the  dark 
as  to  the  reliability  of  the  weapon  system  until  development 
is  completed.  In  order  to  insure  that  the  weapon  system  reli- 
ability will  meet  the  contractual  requirement,  the  program 
manager  must  have  some  type  of  yardstick  by  which  he  can 
visually  measure  the  reliability  growth  of  the  weapon  system 
during  development.  This  yardstick  must  also  provide  inform- 
ation as  to  when  design  changes  are  required,  and  when  the 
system  has  met  its  contractual  requirement.  Barlow  states: 
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"It  is  common  practice,  during  the  development  of  a system 
to  make  engineering  changes  as  the  program  develops.  These 
changes  are  generally  made  in  order  to  correct  design  defici- 
encies and,  thereby,  to  increase  reliability.  This  elimina- 
tion of  design  weaknesses  is  what  we  mean  by  reliability 
growth."  [17] 

To  summarize  the  above,  reliability  growth  is  that  growth 
observed  in  reliability  as  the  weapon  system  progresses  along 
its  life  cycle.  A visual  representation  is  presented  in 
Figure  1. 


B.  THE  YARDSTICK 

Present  policy  requires  that  a reliability  program  be 
established  for  a new  weapon  system  in  accordance  with  MIL- 
STD  785A.  This  military  standard  provides  for  a sound  reli- 
ability program  for  the  attainment  of  the  reliability  require- 
ment. However,  it  does  not  provide  the  means  to  monitor  or 
measure  the  reliability  growth  of  the  weapon  system.  It  is 
proposed  that  a "Reliability  Growth  Curve  Prediction"  be  con- 
tractually imposed  as  part  of  the  Reliability  Program  Plan  to 
provide  the  means  to  monitor  and  measure  reliability. 


C.  RELIABILITY  GROWTH  CURVE 

1.  What  is  It?  Amstradter  states: 

"...reliability  growth  is,  in  reality,  a discrete  function 
which  occurs,  hopefully,  each  time  a design  change  is 
made.  [16] 

2.  Is  It  New? 

Reliability  growth  is  not  new.  Much  has  been  written 
about  it,  as  can  be  noted  in  Appendix  A,  Bibliography  on 
Reliability  Growth.  It  has  only  been  in  recent  years  that  it 
has  come  into  its  own  and  has  been  recognized  as  a useful 
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management  tool.  Although  the  major  emphasis  has  been  only 
to  monitor  reliability  growth  during  development  testing, 
more  thought  is  being  given  to  use  it  as  a forecasting 
method. 

A reliability  growth  model  is  an  analytic  tool  useful 
in  determining  the  time  required  to  develop  an  acceptable 
product.  The  model  monitors  the  progress  of  the  develop- 
ment program  and  can  be  used  to  forecast  the  time  required 
to  achieve  the  reliability  goal  if  the  program  were  to  be 
modified.  [18] 

This  technique  should  be  useful  to  the  reliability 
engineer  in  the  monitoring  of  development  progress  to 
insure  that  reliability  specification  requirements  are 
achieved.  A further  desirable  characteristic  would  be 
for  techniques  to  provide  for  continuing  extrapolation 
from  current  status  to  some  future  result...  This  tech- 
nique is  known  as  reliability  growth  curve  methodology.  [19] 

3 . Two  Types  of  Growth  Curves 

Reliability  growth  curves  can  be  classified  into  two 
categories;  Assessment  and  Apportionment  [20]. 

a.  Assessment — When  reliability-growth  equations  and 
curves  are  prepared  after  a program  is  well  into  the  develop- 
ment phase  and  thereby  utilize  actual  test  data  to  arrive  at 
appropriate  numerics,  growth  analysis  becomes  primarily  an 
assessment  process. 

b.  Apportionment — If  a growth  curve  is  prepared  to 
depict  expected  levels  of  reliability  achievement  and  to  pro- 
vide a continuum  of  time-oriented  goals,  it  can  be  classed  as 
an  apportionment  procedure. 

4.  Models 

There  are  several  reliability  growth  models,  each  of 
which  is  valid  in  its  own  right.  The  purpose  of  this  paper 
is  not  to  present  the  various  models  and  how  to  apply  them, 
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but  to  show  that  reliability  growth  curves  have  a place  in 
management.  However,  to  acquaint  the  reader  with  reliability 
growth  models.  Appendix  B provides  a review. 

5 . The  Purpose  of  Reliability  Growth  Models 

The  use  of  reliability  growth  curves  has  several 
potential  purposes.  They  include: 

a.  Monitoring  the  reliability  progress  of  the  system 
as  it  proceeds  through  development. 

b.  If  a reliability  growth  curve  was  predicted,  it 
provides  a means  to  measure  the  actual  growth  of  the  system. 

c.  It  provides  for  forecasting  of  reliability  in  the 
short  term. 

d.  Provides  the  means  to  measure  the  effectiveness 
of  design  changes. 

e.  Aids  in  defining  design  problems  and  what  their 
impact  on  reliability  will  be. 

f.  Aids  in  the  allocation  and  reallocation  of  resources 
to  achieve  requirements  on  schedule  and  within  constraints . 

g.  Aids  in  determining  requirements  for  future 
procurements . 

h.  Provides  a useful  communication  vehicle  for  high 
level  management. 

i.  Aids  in  planning  the  development  program  and  to 
control  its  progress  as  the  design  matures. 

6 . Is  It  Applicable? 

The  best  method  to  show  the  applicability  of  reli- 
ability growth  curves  is  to  cite  several  examples.  The 
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examples  will  be  divided  into  the  two  categories  of  reliability 
growth  modeling;  assessment  and  apportionment, 
a.  Assessment  Examples 

(1)  F- 15 A Aircraft  [21].  The  reliability  program 
for  the  F15A  did  not  call  for  reliability  growth  modeling. 

The  reliability  requirements  were  typical  of  today's  policies, 
i.e. , MTBF  and  a reliability  program  plan.  As  part  of  the 
reliability  program,  data  were  collected,  analyzed  and  plotted. 
This  plotted  data  (Figure  2)  shows  the  reliability  growth  of 
the  aircraft  during  development  testing. 

It  should  be  noted  that  the  growth  curve  is 
not  continuous  from  Category  I (contractor  evaluation)  to 
Category  II  (Air  Force  evaluation) . The  major  contributing 
factor  for  this  discontinuity  is  the  change  in  operational 
environment.  This  is  one  of  the  considerations  that  must  be 
recognized  in  using  reliability  growth  modeling. 

(2)  TOW  Weapon  System  [22].  The  TOW  weapon  sys- 
tem is  a heavy  assault  weapon  for  the  U.  S.  Army.  The  reli- 
ability requirements  for  this  program  were  based  on  present 
day  policies.  However,  a strong  emphasis  was  stressed  in  the 
"find  and  fix"  and  "overstress"  test  approach  to  reliability. 
Although  reliability  growth  modeling  was  not  a contractual 
requirement,  a plot  of  reliability  growth  was  maintained 
throughout  the  program  (Figure  3)  . 

(3)  F-111A  Aircraft  [23].  The  reliability  pro- 
gram for  the  U.  S.  Air  Force  F-111A  aircraft  was  conducted 
under  present  day  policies.  To  assure  that  the  requirements 
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were  obtained,  a rigorous  reliability  program  was  imposed  on 
the  contractor.  Reliability  growth  prediction  was  not  part 
of  this  program;  however,  data  were  plotted  to  show  the  reli- 
ability growth  of  the  aircraft  during  its  development  (Figure 
4)  . 

b.  Apportionment  Examples 

(1)  C-141  Aircraft  [24].  The  U.  S.  Air  Force 

reliability  requirement  for  the  C-141  was  as  follows:  The 

contractor  shall  plan  and  conduct  a comprehensive  reliability 
program  in  accordance  with  the  requirements  of  Weapon  Systems 
Specification  MIL-R-26674,  Reliability  Requirements  for  Wea- 
pon Systems. 

Although  the  Air  Force  did  not  specify  a 
numerical  reliability  requirement,  the  contractor  established 
a design  goal  and  performed  a prediction  analysis  to  show 
projected  growth  to  this  goal.  As  the  program  progressed, 
predictions  were  continually  updated  which  resulted  in  adjust- 
ment to  the  growth  curve  (Figure  5) . As  the  program  progressed 
through  development,  observed  growth  was  plotted  against  the 
predicted  growth  curve  (Figure  6) . This  visual  representa- 
tion served  as  a basis  for  program  reviews  and  direction  for 
the  contractor  and  the  Air  Force. 

(2)  SATCOM  Terminals  [25].  Reliability  growth 
prediction  was  not  imposed  as  a contractual  requirement,  but 
the  Army  used  the  Duane  and  Weibull  reliability  growth  models 
to  make  reliability  management  decisions.  These  growth 
models  helped  to  plan  the  growth  of  reliability  in  the  develop- 
ment program' and  to  control  its  progress  as  the  design  matured. 
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C-141A  RELIABILITY  PREDICTIONS 
FIGURE  5 


1965  1965  1966 

C-141A  RELIABILITY  GROWTH 
FIGURE  6 


The  authors  of  this  article  summed  up  the  use  of  this  technique 
in  the  following  statement: 

"The  Duane  and  Weibull  growth  curves  were  used  to  deter- 
mine requirements  for  future  buys,  provide  management 
visibility  for  reliability  and  make  decisions  regarding 
test  termination." 

(3)  SM-2  Missile  [26].  To  meet  the  contractual 
MTBF  requirement,  the  contractor  established  a three-phase 
reliability  program.  One  phase  of  this  program  is  "Reliabil- 
ity Growth  Monitoring."  The  procedure  that  the  contractor 
set  up  to  accomplish  this  phase  was  to  plot  a predicted  growth 
curve  that  the  system  was  expected  to  take  during  its  develop- 
ment. The  observed  data  will  be  plotted  against  this  predic- 
tion to  provide  the  impetus  for  additional  action  if  it  appears 
that  the  MTBF  requirement  might  not  be  met.  This  approach  is 
being  taken  by  the  contractor  mainly  due  to  another  program 
in  which  the  MTBF  growth  was  plotted  and  found  to  be  a useful 
management  tool.  In  a conversation  with  J.  C.  Bear  [27],  he 
commented  that  General  Dynamics  is  finding  this  technique  very 
useful  and  feels  that  the  potential  is  there  for  a very  power- 
ful management  tool. 

D.  THE  MANAGEMENT  TOOL 

It  has  been  shown  that  a potential  management  tool  exists 
in  the  use  of  Reliability  Growth  Curves.  Although  the  major- 
ity of  the  interest  to  date  has  been  the  assessment  of  reli- 
ability growth,  interest  is  beginning  to  stir  in  its  use  as 
an  estimator  of  reliability  growth  of  a weapon  system.  This 
is  evident  by  contractual  requirements  now  being  imposed  by 
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the  U.  S.  Army  on  new  weapon  system  developments,  such  as  their 
Stinger  program  [28]. 


Present  day  policies  do  not  provide  the  program  manager 
visibility  as  to  the  reliability  growth  of  the  system.  The 
manager  does  not  become  aware  that  the  system,  in  its  present 
design,  will  not  meet  the  requirements  until  it  is  well  into 
demonstration  testing.  With  a predicted  reliability  growth 
curve,  the  program  manager  has  the  means  by  which  to  measure 
actual  growth  and  to  aid  in  decision  making  as  to  design 
changes  and  testing  of  the  system. 

E.  IMPLEMENTATION 
1.  Requirement 

The  requirement  for  a "Reliability  Growth  Curve 
Prediction"  should  be  part  of  the  Reliability  Program  Plan 
requirement  of  the  RFP . The  requirement  could  take  one  of 
two  forms : 

a.  The  Navy  with  their  knowledge  of  the  new  weapon 
system  design  and  development  time  constraints,  could  submit 
the  desired  Reliability  Growth  Curve  to  which  the  contractor 
could  respond  as  they  presently  do  with  the  MTBF  requirement. 

b.  The  Navy  could  require  that  the  contractor  submit 
a "Reliability  Growth  Curve"  as  part  of  their  reliability  pro- 
gram plan. 

In  either  case,  because  of  its  sensitivity  depending  upon  the 
model  used  and  the  factors  involved  in  predicting  the  curve, 
the  final  Reliability  Growth  Curve  Prediction  will  be  a nego- 
tiated item  between  the  Navy  and  the  contractor. 
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2.  Pit-Falls 


In  that  Reliability  Growth  Curve  Prediction  would  be 
a new  management  technique,  pit-falls  should  be  expected  in 
the  early  stages.  All  factors  that  affect  the  prediction  and 
the  exact  sensitivity  to  the  different  models  can  only  be 
learned  with  use.  The  following  is  a listing  of  several  fac- 
tors that  must  be  considered  in  selecting  the  appropriate 
model  and  the  use  of  Reliability  Growth  Curve  Prediction. 

a.  Complexity  of  the  weapon  system  in  that  it  will 
have  a definite  bearing  on  expected  failure  rate. 

b.  Schedule  time  alloted  for  development  of  the  sys- 
tem. (Short  time,  you  would  expect  to  see  a rapid 
growth;  long  time,  a more  gradual  growth.) 

c.  The  various  environments  that  the  system  is  sub- 
jected to  during  development.  (Laboratory,  con- 
tractor field  testing,  customer  testing,  etc.) 

d.  Definition  of  failure:  To  avoid  discontinuities 

in  the  growth  curve,  failure  definition  must  remain 
constant  [27] . 

e.  Number  of  expected  failures:  This  would  have  a 

bearing  in  selecting  the  model  that  would  provide 
the  best  growth  fit  for  the  system. 


0 
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V.  CONCLUSIONS 


The  program  manager,  today,  has  no  means  available  to  him 
to  provide  visibility  with  regards  to  reliability  growth. 
Reliability  prediction  provides  the  manager  with  the  potential 
maximum  value  at  that  point  in  time  when  the  prediction  is 
made.  Reliability  demonstration  provides  the  manager  with  the 
actual  value  at  the  time  when  the  test  was  conducted.  Neither 
of  these  two  methods  provides  the  manager  with  the  answer  to: 
"Is  the  weapon  system  going  to  meet  the  reliability 
requirement?" 

The  "Reliability  Growth  Curve  Prediction"  would  provide 
the  manager  with  insight  as  to  how  the  weapon  system  should 
meet  the  reliability  requirement.  And  when  actual  data  are 
plotted  and  compared  to  the  predicted  curve,  it  would  provide 
the  manager  insight  to  the  question:  "Is  the  weapon  system 

going  to  meet  the  reliability  requirement?" 
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APPENDIX  B 


A REVIEW  OF  SOME  RELIABILITY  GROWTH  MODELS* 

This  appendix  describes  a number  of  reliability  growth  models  which 
are  currently  available.  Each  model  is  briefly  described  including  the 
basic  assumptions  that  were  made  in  deriving  the  models.  Technical 
references  are  given  for  each  of  these  models  where  a more  complete 
discussion  of  the  model  may  be  found. 


Model  1.  Lloyd  and  Lipow  (see  Appendix  B,  Reference  28)  introduced  a 
reliability  growth  model  for  a system  which  has  only  one  failure  mode. 
For  each  trial  it  is  assumed  that  the  probability  is  a constant  that 
the  system  will  fail  if  the  failure  mode  has  not  been  previously 
eliminated.  If  the  system  does  not  fail,  no  corrected  action  is  per- 
formed before  the  next  trial.  If  the  system  fails,  then  an  attempt  is 
made  to  remove  the  failure  mode  from  the  system.  The  probability  of 
successfully  removing  the  failure  mode  is  also  assumed  to  be  a constant 
for  each  attempt.  They  show  that  the  system  reliability,  R^,  on  the 
n-th  trial  is 


R = 1-Ae 
n 


-C(n-l) 


where  A and  C are  parameters. 


Model  2.  Another  reliability  growth  model  was  considered  by  Lloyd  and 
Lipow  (see  Appendix  B,  Reference  28)  where  the  development  program  is 
conducted  in  X stages  and  on  the  i-th  stage  a certain  number  of  systems 
are  tested.  The  reliability  growth  function  considered  was 


R.  = R -a/i, 

where  R^  is  the  system  reliability  during  the  i-th  stage,  R^  is  the 
ultimate  reliability  as  i->=>  and  a>0  is  a parameter.  Maximum  likelihood 
and  least  squares  estimates  of  3na  a are  given  by  Lloyd  and  Lipow 

* Ref:  Crow,  Larry  H.  Reliability  Growth  Modeling,  US  Army  Material 

System  Analysis  Activity,  Aberdeen  Proving  Ground,  MD,  Tech  RDt  55,  1972. 


along  with  a lower  confidence  limit  for  R^.. 


Model  3.  Weiss  (see  Appendix  B,  Reference  41)  considered  a reliability 
growth  model  where  the  mean  time  to  failure  of  a system  with  exponential 
life  distribution  is  increased  by  removing  the  observed  failure  nodes. 

In  particular,  he  showed  that  when  certain  conditions  hold,  the  increase 
of  the  mean  time  to  failure  is  approximately  at  a constant  percent  per 
trial.  That  is,  if  6(i)  is  the  mean  time  to  failure  of  the  system  at 
trial  i then  Q(i)  may  be  approximated  under  certain  conditions  by 

6(i)  = Ae*"'*, 

where  A and  C are  parameters.  Note  that 

Q(i+1)  = eC6(i). 

The  maximum  likelihood  estimates  of  A and  C are  given  by  Weiss. 

Model  4.  Wolman  (see  Appendix  B,  Reference  42)  considered  a situation 
where  the  system  failures  are  classified  according  to  two  types.  The 
first  type  is  termed  "inherent  cause"  and  the  second  type  is  termed 
"assignable  cause".  Inherent  cause  failures  reflect  the  state-of-the 
art  and  may  occur  on  any  trial  while  assignable  cause  failures  may  be 
eliminated  by  corrective  action,  never  to  appear  again.  Wolman  assumed 
that  the  number  of  original  assignable  cause  failures  is  known  and  that 
whenever  one  of  these  modes  contribute  a failure,  the  mode  is  removed 
permanently  from  the  system.  Wolman  uses  a Markov-chain  approach  to 
derive  the  reliability  of  the  system  at  the  n-th  trial  when  the  failure 
probabilities  are  known. 

Model  5.  Barlow  and  Scheuer  (see  Appendix  B,  Reference  3)  considered 
a nonparametric  model  for  estimating  the  reliability  of  a system  during 
a development  program.  They  assumed  that  the  design  and  engineering 
changes  do  not  decrease  the  system's  reliability,  but,  unlike  some 
other  models,  they  do  not  fit  a prescribed  functional  form  to  the 


44 


reliability  growth.  Their  model  is  similar  to  V/olman's  in  that  each 
failure  must  be  classified  either  as  inherent  or  assignable  cause. 

It  is  further  assumed  that  the  development  program  is  conducted  in 
K stages,  with  similar  systems  being  tested  within  each  stage.  For 
each  stage,  the  number  of  inherent  failures,  the  number  of  assignable 
cause  failures  and  the  number  of  successes  are  recorded.  In  addition, 
they  assumed  that  the  probability  of  an  inherent  failure,  qQ,  remains 
the  same  throughofct  the  development  program  and  that  the  probability  of 
an  assignable  cause  failure,  q^,  in  the  i-th  stage  does  not  increase 
from  stage  to  stag©  of  the  development  program.  The  authors  obtained 
the  maximum  likelihood  estimates  of  qQ  and  of  the  q^'s  subject  to  the 
condition  that  they  be  nonincreasing.  A conservative  lower  confidence 
bound  for  the  reliability  of  the  system  in  its  final  configuration  was, 
also,  given. 

Model  6.  Virene  (see  Appendix  B,  Reference  38)  considered  the  suitability 
of  the  Gompertz  equation 


t 

R = abC  , 

0<b<l,0<c<l,  for  reliability  growth  modeling.  In  this  equation 
a is  the  upper  limit  approached  by  the  reliability  R as  the  development 
time  t-*».  The  parameters  a,  b and  c are  unknown.  Virene  gave  estimates 
of  these  parameters  and  demonstrated  by  examples  the  application  of  this 
model. 

Model  7.  Duane  (see  Appendix  B,  Reference  17) considered  a deterministic 
approach  to  reliability  growth  modeling.  He  analyzed  data  available 
for  several  systems  developed  by  General  Electric  in  an  effort  to 
determine  if  any  systematic  changes  in  reliability  improvement  occurred 
during  the  development  programs  for  these  systems.  His  analysis  revealed 
that  for  these  systems,  the  cumulative  failure  rate  versus  cumulative 
operating  hours  fell  close  to  a straight  line  when  plotted  on  log-log 
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paper.  The  cumulative  failure  rate  appeared  to  decrease  at  approximately 
the  -0.4  or  -0.5  power  of  cumulative  operating  hours. 


The  types  of  systems  investigated  were  of  the  complex  electro- 
mechanical and  mechanical  nature.  Duane  concluded  that  a line  with  a 
slope  of  -0.5  representing  cumulative  failure  rate  as  a function  of 
cumulative  operating  hours  on  log- log  paper  would  probably  be  suitable 
for  reflecting  reliability  growth  for  similar  type  systems  developed  at 
General  Electric. 

Mathematically,  Duane's  failure  rate  equation  may  be  expressed  by 

A(T)  = KT"a, 

K > 0,  0 <_  a <_  1,  where  A (T)  is  the  cumulative  failure  rate  of  the 
system  at  operating  time  T,  and  K and  a are  parameters.  It 
follows  then  that 

A(T)  - 

T 

where  E(T)  is  the  expected  number  of  failures  the  system  will 
experience  during  T units  of  operation.  This  yields 

ECT)  = KT1-a. 


With  this  notation  a =■  0.5  closely  represented  the  types  of  systems 
considered  by  Duane. 

Model  8.  Pollock  (see  Appendix  B,  Reference  32)  considered  a Bayesian 
reliability  growth  model  for  a system  undergoing  development.  The 
parameters  of  the  model  are  assumed  to  be  random  variables  with 
appropriate  prior  distribution  functions.  Using  his  results,  one  may 
project  the  system  reliability  to  any  time  after  the  start  of  the 
development  program  without  data  and,  also,  estimate  the  system 
reliability  after  data  have  been  observed.  He  further  gave  precision 
statements  regarding  the  projection  and  estimation. 

Model  9.  Barlow,  Proschan  and  Scheuer  (see  Appendix  B,  Reference  2) 
considered  a reliability  growth  model  which  assumes  that  a system  is 
being  modified  at  successive  stages  of  development.  At  stage  i the 
system  reliability  (probability  of  success)  is  p^.  The  model  of 
reliability  growth  under  which  one  obtains  the  maximum  likelihood 
estimates  of  P1,P2»»..»?K  assumes  that 

Pi  -P2  - -?K* 

That  is,  it  is  required  that  the  system  reliability  be  not  degraded 
from  stage  to  stage  of  development.  No  particular  mathematical  form 
of  growth  is  imposed  on  the  reliability.  In  order  to  obtain  a con- 
servative lower  confidence  bound  on  p^.,  it  suffices  to  require  only 
that 

p..  > max  p.  . 
l<k 

That  is,  it  is  only  necessary  that  the  reliability  in  the  latest  stage 
of  development  be  at  least  as  high  as  that  achieved  earlier  in  the 
development  program. 

Data  consist  of  x^  successes  in  n ^ trials  in  stage  i,  i=l,...,K. 
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A variation  of  this  model  is  treated  in  Barlow  and  Scheuer  (see 
Model  5).  In  that  model  two  types  of  failure,  inherent  and  assignable 
cause,  are  distinguished. 

Model  10.  Another  reliability  growth  model  considered  by  Barlow, 
Proschan  and  Scheuer  (see  Appendix  c.  Reference  2)  assumed  that  at 
stage  i of  development  the  distribution  of  system  life  length  is  F^. 
The  model  of  reliability  growth  under  which  the  maximum  likelihood 
estimates  of  F^  (t)  ,F2 (t) , . . . ,Fj.(t)  are  obtained,  writing 

= l-F.(t) 


is 

Pj(t)  1 F2(t)  1 ...  1 FK(t) 

for  a fixed  t >_  0.  In  order  to  obtain  a conservative  upper  confidence 
curve  on  F^ft)  and  thereby,  a conservative  lower  confidence  curve  on 
Fj,(t)  for  all  non-negative  values  on  t,  it  suffices  only  to  require 
that 


F 


K 


(t)  >_  max  F.  (t) 
i<K  1 


for  all  t >_  0.  That  is,  the  probability  of  system  survival  beyond  any 
time  t in  the  latest  stage  of  development  is  at  least  as  high  as  that 
achieved  earlier  in  the  development  program. 

Data  consist  of  independent  life  length  observations 


Model  11.  Barlow,  Proschan  and  Scheuer  (see  Appendix  C,  Reference  2), 
also,  considered  a reliability  growth  model  which  assumes  that  the 
system  life  at  the  i-th  stage  of  development  has  increasing  failure  rate 
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Because  of  improvement  from  stage  to  stage 


r!  (t)  >_  r2(t)  1 ...  1 rK(t) 

for  t >_  0,  where  r^t)  is  the  failure  rate  at  time  t at  the  i-th  stage 
of  development.  That  is,  for  each  t >_  0,  the  probability  of  system 
failure  in  the  interval  (t,  t+dt) , given  survival  till  time  t,  does  not 
increase  from  stage  to  stage  of  the  development  program. 

Given  life-length  observations,  ,X^2> " * * ’^in’  t*ie  max^mum 
likelihood  estimates  of  r, (t) ,r2 (t) , . . . ,rK(t)  are  obtained. 
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